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Summary 


Legacy Full-Featured DBMS 


A simple in-house 
database engine 
tailored to MMOGs is 

highly effective to provide 
mobile MMOGs for multi- 
millions unique users. 



Many Business Functions 


Authentication 


Security 

Encryption 

Transaction 

Data Mining 


Full SQL 



Minimal & Tailored DBMS 




Simplified Relational Access 


Z 


Lock-Free I/O Operations 
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Game developers do not 
use almost of those 
business-oriented 
functions 


It is easy to implement 
the minimal database 
management system. 


Legacy Full-Featured DBMS 


Many Business Functions 



Authentication 


Security 

Encryption 

Transaction 

Data Mining 


Full SQL 


Minimal & Tailored DBMS 






Simplified Relational Access 


I 


Lock-Free I/O Operations 



My message is: 


Lfl l your MM 

database engine 
fcaitetwite your iillMi 
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Quick Introduction 


Who are you? 
What is Cygames? 
What is problem? 
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Introduction: Who I am 


Shuichi Kurabavashi, Ph.D. 



Technical advisor of Cygames, Inc. 
Also Director of Cygames Research 

Project Associate Professor at the 
Graduate School of Keio University 
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A 


Introduction: Cygames is one of the 
largest mobile game developers in Japan 





Known as the 
developer of the wildly 
popular card battle 
game "Rage of 
Bahamut". 

Recently released 
"Shadowverse". 




The number of 
registered users 
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Background: We have been providing 
mobile MMOGs for 10 millions users. 
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Background: Mobile gaming is an 
important subject of DB Research 


Tokyo Stock 
Exchange 



6,700/sec 

(400,000/min) 

transactions 


Twitter 



40,000/sec 
messages at 
peak. 



Mobile game 
(Japan) 



100,000/sec 
transactions 
in average. 

V J 
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Problem Definition 

• Modern MMOGs require 
databases to support large- 
scale availability , strong 
consistency , and real-time 

response. 

• It is difficult to support such 
capabilities efficiently by 
using conventional systems. 


MMOG 



availability 


consi sten cy 
real-time 



nvep 
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CAP Theorem 


C: Consistency of data 

• After data has been updated, if 
something else references that data, 
it will always be guaranteed that the 
updated data can be referenced. 

A: Availability of the system 

• No matter what the current 
situation, the system will continue to 
operate. 

P: Tolerance to network 
partitions 


Data can be distributed. 



http://berb.github.io/diploma-thesis/original/ 061 _challenge.html 
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CAP Theorem 


C: Consistency of data 

• After data has been updated, if 
something else references that data, 
it will always be guaranteed that the 
updated data can be referenced. 

A: Availability of the system 

• No matter what the current 
situation, the system will continue to 
operate. 

P: Tolerance to network 
partitions 


Out of these three 
guarantees, only two can 
be fulfilled at a time. 


Data can be distributed. 



http://berb.github.io/diploma-thesis/original/ 061 _challenge.html 
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Choosing Appropriate 
System 


CA: conventional 
relational 
database 
systems, such as 
MySQL, Oracle, 
and SQL server, 
strong 
consistency 



CP: Apache 
Hbase, Hadoop, 
BigTable, 

Good for 
analytics 


http://berb.github.io/diploma-thesis/original/061_challenge.html 


AP: SNS,NoSQL, 

eventual 

consistency 
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CAP Theorem 


CA is the most 
important for 
games, because 
games require 

strong 

consistency and 
fast response 



CP: Apache 
Hbase, Hadoop, 
BigTable, 

Good for 
analytics 


http://berb.github.io/diploma-thesis/original/ 061 _challenge.html 


AP: SNS,NoSQL, 

eventually 

consistency 




Typical Backend Architecture 
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Cluster Set 3 
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Cluster Set 32 


Analytics 

Layer 


Data Mining 
Environment 


Customer 
Support DB 


Data Warehouse Systems and Statistical Analysis Systems 
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Primitive Unit 


Main memory, memcached 
stores static data shared 
among all the PHP processes. 


MySQL stores status data 
which are updated in a 
real-time manner. 


Partitioning 


Web Server 




memcached 


This bandwidth is the 
most precious 
resources. 



MySQL 

toaster ^ 


MySQL 

UsIaveU 


MySQL 
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I Fusion-io 
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Cluster Set 1 

iiii 



Combination of shardinq (horizontal 
decomposition) and partitioning (vertical 
decomposition) brings high-level parallelism 
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Data mining process including JOIN operations is carried 
out by replica DBs that are replicated from the master 
database asynchronously. 

Sharding 
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Layer 
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Data Warehouse Systems and Statistical Analysis Systems 
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MySQL is working now. But... 


Operation can be highly inefficient 
and unprofitable 


Not so Large Capacity per 

scalable maintenance machine is 

cost not so high 

1 1 1 1 





RDB functionalities are appropriate for 
C-A requirements, but it is too fat. 


3 Overheads that decrease RDB's performance 

\ 

Overhead-1: Too Generic Structure 

s > 

\ 

Overhead-2: Not utilizing data access pattern in games 
/ 


\ 

Overhead-3: Not utilizing modern hardware 

s > 






Overhead-1: 

Too Generic Structure! 


Data Model 
supported 
by Apps 


External 

Schema-1 


External 

Schema-2 


External 

Schema-^ 


fcJUA 



Data Model 
supported by DBMS 


•jfe m 


Data Model 
supported by storage 


Conceptuc 

3 l Schema 




r 

Internal Schema 



CREATE TABLE (...) ; 


TupleTableSlot *rec; 
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Overhead-1 :Too Generic Structure! 


Planner (logical optimization) 

• Rewrites queries for better 
performance, by analyzing 
queries as relational calculus. 

Executor (physical 

optimization) 

• Executes query primitives such 
as relational algebras. 

Storage Subsystem 

• Reads and writes disk storages. 




Let's resolve Overhead-2: 

Not utilizing data access pattern in games 



Insert 

Update 

Select 

Real-time 

Consistency 

Query Type 

Legacy Web 
(e-commerce) 

Small 

Small 

Large 

No 

Strong 

Consistency 

Dynamic 

(mutable) 


SNS(Large Scale 
Web Apps) 

Large 

Small 

Massively 

Large 

No 

Eventual 

Consistency 

Dynamic 

(mutable) 

IoT (Stream DB) 

M 

Massively 

Large 

no 

Small 

Yes 

Application- 

Dependent 

Static 

(immutable) 

Mobile Game 

1 Small 

Massively 

Large 

Massively 

Large 

Yes 

Strong 

Consistency 

Static 

(immutable) 


We faces technical requirements that are essentially 
different from a conventional DBs 


mi 
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Let's resolve Overhead-3: 

Not utilizing modern hardware 


The throughput performance becomes from 
10 to 20 times faster, due to the multi-core 
CPU with the high parallelism and a SSD 
with the highly parallel access. 



Solution 


r 


v 


Query Pre-Compilation 
(Immutable Query) 



A 


y 


r n 

Lower Footprint 
query processing 


f 

Lock- Free Thread 
Scheduling 


A 


J 


■ III 




GDC 


GAME DEVELOPERS CONFERENCE EUROPE COLOGNE, GERMANY • 15-16 AUGUST 2016 



Static Query Analysis is effective 

If we can know all the query to be processed, 
we can apply holistic optimization by 
analyzing their data access pattern and 
implicit race condition among them. 





CySQL 


Immutable 
Queries (Statically 
analyzed and 
compiled queries) 




■ III 
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Legacy Query Model 


• Legacy query model cannot optimize 
holistic query, because queries are 
submitted dynamically. 




App 

Legacy DB 


Server 



Immutable Query 

Immutable query model requires the all 
queries to be defined at deploy time 

, — Deploy-Time , ( Runtime 





CySQ L 

Immutable 
Queries (Statically 
analyzed and 
compiled queries) 
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SQL parsing and optimizing takes large cost 



Controlling transaction manager directly from 
the application, we can achieve nearly 10 
times faster performance than SQL 


Planner 


Executer 


v.. 


We can bypass the planner 
and the executor, by 
compiling SQL into 
executable machine code. 


Storage 


V 


7 




Query Processor Bypassing 


Introducing query pre-compilation and 
bypassing planner and executor 



> 

i c \ 


SQL 

^1 executable 


SELECT * 

|Ht 0100100100101 


FROM 

rT| 0011101010010 


table; j 

' V 011101100011 > 


SQL pre-compilation 



Bypassing SQL modules 


mi 
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Lock-Free Transaction Processing 

Legacy RDB embraces a lot of overhead of sync inside the 
DB engine for lock mechanism. Each of those overheads 
is small, but small overhead will be accumulated, so each 
overhead is the cost which can't be ignored. 


Snapshot 
creation 
for MVCC 



Row-level lock 



Buffer lock 
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Can we realize Lock-Free? 


By using the thread scheduling specialized 
in the data access pattern specific to the 
game, multiple threads read/writes 


databases without locks. 



• race condition, 

• dependency, 

• priority 
between queries 



GDC 


GAME DEVELOPERS CONFERENCE EUROPE COLOGNE, GERMANY • 15-16 AUGUST 2016 



Basic Concept of Lock-Free Thread Scheduling 


Query- 1 
Job Queue 


Query-2 
Job Queue 


Query-3 
Job Queue 


Query-n 
Job Queue 



■ III 


In the immutable query model, DB 
engine can schedule threads perfectly 
without synchronization lock. 

• Because the DB engine can analyze race 
condition between queries statically. 

• The thread scheduling strategy is easy: 
"Do not schedule the queries that cannot 
be executed simultaneously." 

1 1 1 1 1 
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The number of CPU cores are increasing for 
both energy and performance efficiency. 
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Parallelism in I/O Devices 


Not so parallel 



Highly Parallel 

■ III 
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Parallelism in I/O Devices 



Not so parallel 




Highly Parallel 

■ III 


Throughput (sec) 
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Comparison of the prototype and MySQL 
UPDATE command 
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150 
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0 


Executing UPDATE operations by using a 
randomly generated value. 



6.3 



500000 
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Update Count 


Prototype ■ MySQL 







10.4 


2000000 







Theoretical Conclusion 


Increase the number 
of threads as many 
as you can 
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Implementation! 


• Here I introduce a quick 
implementation method of read 
only database engine. 
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Use Case: 
Fast 
Search 
Engine 


loqical disiunction 


Forestcraft Swordcraft Runecraft Dragoncraft Shadowcraft Bloodcraft Havencraft Neutral 


'8 '9 M ( Clear 


Followers ■ Spells 


Amulets Clear 


Legendary clear 


Advanced Search ▲ 


Search 


logical conjunction 
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A 




Query type 
detection 

S HAD QW.VE RSJE 

P 0 * A L 

NEWS 

CARD LIST 

DECK BUILDER 

£ Sign In 

HOME > Card List 






Card Name [inter 






Class 






^ I 

. w 1 - x* 

Lftiu-A 1 rRIKA^ v> -JSj| 


L In 

HKTi 

i<rebi.wiwi 

1 ^ i 

f Clear } 


Forestcraft Swordcraft 

Runecran: 

L/i ayui k.i a i l 

3iiaJwwi.iarL Diooacratt Havencraft 

Neutral 


Cost *2 *2 

# g *4 


9 !() C Clear ^ 



Type 

■ Followers 

■ Spells 

■ Amulets 

Clear 


Rarity 

■ Bronze 

■ Silver 

■ Gold 

■ Legendary 

Clear 


Those are sub-queries corresponds to the immutable 
queries 
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A 




Query type 
detection 

S HAD QW.VE RSJE 

P 0 * A L 

NEWS 

CARD LIST 
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£ Sign In 
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Card Name [inter 






Class 
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f Clear } 


Forestcraft Swordcraft 

Runecran 

L/i ayui k.i a i l 

3iiadwwi.iarL Diooacratt Havencraft 
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Cost *2 *2 

# g *4 


9 !() C Clear ^ 



Type 

■ Followers 

■ Spells 

■ Amulets 

Clear 


Rarity 

■ Bronze 

■ Silver 

■ Gold 

■ Legendary 

Clear 


When the target data is fixed, we can execute those 
sub queries before the runtime. 
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Data Structure 




Each block represents an 
array that stores results 
of the specific queries 
applied to the specific 

attributes. 

Query 
1 Results 


ii 



> Queries 






Conclusion 

• You can implement tiny RDB 
within a hour. 


Increase the number of 
threads as many as you can 
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